The jury is in: Linux is the benchmark for open-source software quality, according to a study into defects occurring in the software development process. The study was started in partnership with Homeland Security Department, but is now managed by Coverity.
The finding is based on an analysis by the Coverity Scan Service, which for more than seven years analyzed 850 million lines of code from more than 300 open-source projects, including those written in Linux, PHP and Apache.
Using a measure of defects per 1,000 lines of code, the study found Linux consistently recorded defect densities of less than 1.0, with versions scanned between 2011 and 2012 having defect rates below 0.7.
Researchers also found high-risk defects were prevalent in the software development process, with 36 percent of defects identified by the firm's 2012 report classified as a "threat to overall software quality and security if undetected."
The most common high-risk defects included memory corruption, illegal memory access and resource leaks, "all difficult to detect without automated code analysis," Coverity reported.
In general, Coverity found the average quality of open-source software was virtually equal to that of proprietary software. Open-source projects showed an average defect density of .69, the study found, a dead heat with the .68 for proprietary code developed by enterprise customers of the service.
Although the average rates of defects in the two types of code are nearly identical, researchers did find a difference in quality trends based on the size of the development project.
For instance, as proprietary software coding projects passed 1 million lines of code, defect density dropped from .98 to .66, a sign that software quality rises in proprietary projects of that size.
That trend reversed itself in the cost of open-source code, researchers found. Open source projects between 500,000 and 1 million lines of code had a defect density of .44, which grew to .75 when those projects went over the 1 million line mark.
Coverity said the discrepancy was caused by "different dynamics between the open source and proprietary development teams" and differences in when the teams "implemented formalized development processes."
